TEXT CLASSIFICATION TASKS

Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even Performance (2025.emnlp-main)

Challenge:	Existing studies on text classification have focused on the bias towards the individuals mentioned in the text content.
Approach:	They propose a framework to mitigate implicit bias in text classification models based on demographic attributes of authors . they propose to use this framework to train deep text classifiers to make predictions on the right features .
Outcome:	The proposed framework outperforms existing models significantly in fairness and performance.

Copied to clipboard

Branislav Pecher, Ivan Srba, Maria Bielikova

Analyzing Biases to Spurious Correlations in Text Classification Tasks (2022.aacl-short)

Challenge:	specialised small models outperform general large models with few labelled samples (on average 100) performance variance is taken into account when comparing the number of labelled sample required to tune a specialised model with a larger number of samples.
Approach:	They find that specialised small models need only few labelled samples to outperform general large models with limited labelled data.
Outcome:	The proposed models outperform general large models with few labelled samples and take performance variance into account.

Copied to clipboard

Adian Liusie, Vatsal Raina, Vyas Raina, Mark Gales

SPEAR : Semi-supervised Data Programming in Python (2022.emnlp-demos)

Challenge:	Often these systems exceed human performance, but there is a caveat: standard benchmarks often assume that training and evaluation data are drawn independently and identically from the same underlying distribution.
Approach:	They propose to exploit spurious correlations in training data to exploit these correlations . they show that even when only ‘stop’ words are available, it is possible to predict the class significantly better than random.
Outcome:	The proposed model can predict class significantly better when only ‘stop’ words are available at the input stage, but can degrade the ability of the system to generalize well to out-of-domain data.

Copied to clipboard

Guttu Abhishek, Harshad Ingole, Parth Laturia, Vineeth Dorna, Ayush Maheshwari, Ganesh Ramakrishnan, Rishabh Iyer

Multimodality

Challenge:	a package for data programming with semi supervision implements several recent approaches to label and train machine learning models.
Approach:	They propose an open-source python library for data programming with semi supervision . the library implements several recent data programming approaches including heuristics and association of noisy labels to training datasets.
Outcome:	The proposed package implements several recent approaches for data programming with semi supervision.

Assessing Social License to Operate from the Public Discourse on Social Media (2020.coling-industry)

Copied to clipboard

Chang Xu, Cecile Paris, Ross Sparks, Surya Nepal, Keith VanderLinden

Challenge:	Social License to Operate (SLO) is the level of support organisations gain from the public.
Approach:	They propose to extract and transform peoples’ stances towards an organisation into SLO levels by performing a chain of three text classification tasks.
Outcome:	The proposed system extracts and transforms peoples’ stances towards an organisation into SLO levels.

Is Sparse Attention more Interpretable? (2021.acl-short)

Copied to clipboard

Clara Meister, Stefan Lazov, Isabelle Augenstein, Ryan Cotterell

Cost-Sensitive BERT for Generalisable Sentence Classification on Imbalanced Data (D19-50)

Challenge:	Sparse attention has been claimed to increase model interpretability . however, the attention distribution is typically over representations internal to the model rather than the inputs themselves .
Approach:	They conduct experiments to understand how sparsity affects our ability to use attention as an explainability tool.
Outcome:	The proposed model does not map to a sparse set of influential inputs, but rather to fewer inputs.

Copied to clipboard

Harish Tayyar Madabushi, Elena Kochkina, Michael Castelle

Effective Adversarial Regularization for Neural Machine Translation (P19-1)

Challenge:	Popular NLP tasks such as sentiment analysis and event extraction from social media are examples of imbalanced classification problems.
Approach:	They propose a method to generalise on dissimilar training and test data using a measure of similarity between datasets.
Outcome:	The proposed method achieves the second highest score on sentence-level propaganda classification.

Copied to clipboard

Motoki Sato, Jun Suzuki, Shun Kiyono

Label Representations in Modeling Classification as Text Generation (2020.aacl-srw)

Challenge:	Existing (small) perturbations that induce a critical prediction error in machine learning models are often referred to as adversarial examples.
Approach:	They propose to use adversarial perturbations to regularize text classification tasks by adding adversarials to a typical NMT model structure.
Outcome:	The proposed method significantly improves performance of NMT models, such as LSTM-based and Transformer-based models.

Copied to clipboard

Xinyi Chen, Jingxian Xu, Alex Wang

XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Models (2023.acl-demo)

Challenge:	Existing methods for text generation use strings to represent labels . linguistic properties of labels do affect performance, though their results are limited to document retrieval.
Approach:	They investigate the effect of string representations on how effectively a model learns a task . they use four standard text classification tasks to model string representation .
Outcome:	The proposed model improves on four standard text classification tasks . the results are largely negative in the low data setting .

Copied to clipboard

Dong-Ho Lee, Akshen Kadakia, Brihi Joshi, Aaron Chan, Ziyi Liu, Kiran Narahari, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren

Bridging CNNs, RNNs, and Weighted Finite-State Machines (P18-1)

Challenge:	Existing models are susceptible to learning spurious biases that do not reflect the underlying task.
Approach:	They propose an open-source framework for explanation-based model debugging that allows users to provide various forms of feedback on model explanations.
Outcome:	The proposed framework improves model’s OOD performance on text classification tasks by up to 18%.

Copied to clipboard

Roy Schwartz, Sam Thomson, Noah A. Smith

Challenge:	recurrent and convolutional neural networks are useful for encoding natural language utterances.
Approach:	They propose a model that combines neural representation learning with weighted finite-state automatas to learn a soft version of traditional surface patterns.
Outcome:	The proposed model is comparable or better than a BiLSTM baseline and a CNN baseline on three text classification tasks.

Universal Language Model Fine-tuning for Text Classification (P18-1)

Copied to clipboard

Jeremy Howard, Sebastian Ruder

Performance-Efficiency Trade-Offs in Adapting Language Models to Text Classification Tasks (2022.aacl-short)

Challenge:	Existing approaches to computer vision require task-specific modifications and training from scratch.
Approach:	They propose a method that can be applied to any task in NLP and propose to open-source it.
Outcome:	The proposed method outperforms the state-of-the-art on six text classification tasks, reducing error by 18-24% on majority of datasets.

Copied to clipboard

Laura Aina, Nikos Voskarides, Roi Blanco

Few-Shot Text Generation with Natural Language Instructions (2021.emnlp-main)

Challenge:	Pre-trained language models (LMs) are state-of-the-art when adapted to text classification tasks.
Approach:	They compare fine-tuning, prompting, and knowledge distillation procedures to train pre-trained language models to downstream tasks.
Outcome:	The proposed training procedures perform better when trained with fine-tuning or prompting on large train sets than when trained by prompting or fine-untun.

Copied to clipboard

Timo Schick, Hinrich Schütze

An Information-Theoretic Approach to Analyze NLP Classification Tasks (2024.acl-long)

Challenge:	Existing approaches to text generation combine task descriptions and examples with supervised learning.
Approach:	They propose a method for text generation that is based on pattern-exploiting training.
Outcome:	The proposed approach improves on several summarization and headline generation datasets.

Copied to clipboard

Luran Wang, Mark Gales, Vatsal Raina

On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choices (2024.emnlp-main)

Challenge:	Natural language processing (NLP) tasks take either a single or multiple text elements to predict an output variable.
Approach:	They propose an information-theoretic framework to analyse the influence of inputs on the output of text classification tasks.
Outcome:	The proposed framework is available at: https://github.com/WangLuran/nlp-element-influence.

Copied to clipboard

Branislav Pecher, Ivan Srba, Maria Bielikova

Text Classification with Negative Supervision (2020.acl-main)

Challenge:	Using limited labelled data, learning with limited labels is sensitive to the effects of uncontrolled randomness.
Approach:	They propose to investigate the effects of individual randomness factors while taking the interactions between them into consideration.
Outcome:	The proposed method mitigates the effects of other factors while observing how the performance varies across multiple runs.

Copied to clipboard

Sora Ohashi, Junya Takayama, Tomoyuki Kajiwara, Chenhui Chu, Yuki Arase

Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation (N19-1)

Challenge:	Existing models for text representations have shown state-of-the-art performance on text classification tasks, however, the discrepancy between semantic similarity of texts and labelling standards affects classifiers.
Approach:	They propose a simple multitask learning model that uses negative supervision to generate distinct representations for texts with different labels.
Outcome:	The proposed model outperforms state-of-the-art models on classification tasks in three different languages.

Copied to clipboard

Radu Tudor Ionescu, Andrei Butnaru

Representation Learning

Challenge:	a novel word embedding representation for text documents is proposed . the method is based on the Vector of Locally-Aggregated Descriptors used for image representation .
Approach:	They propose a novel representation for text documents based on aggregating word embedding vectors into document embeddables.
Outcome:	The proposed representation improves on the Movie Review data set and is 10% better than the state-of-the-art representation.

BERT Goes Off-Topic: Investigating the Domain Transfer Challenge using Genre Classification (2023.findings-emnlp)

Copied to clipboard

Dmitri Roussinov, Serge Sharoff

Training Language Models under Resource Constraints for Adversarial Advertisement Detection (2021.naacl-industry)

Challenge:	Pretrained language models have improved performance of text classification tasks, but they still suffer from spurious domain-specific clues.
Approach:	They propose a method to augment pretrained language models by generating texts in any desired genre and on any desired topic.
Outcome:	The proposed method improves on genre classification tasks while showing no improvement for other topics.

Copied to clipboard

Eshwar Shamanna Girishekar, Shiv Surya, Nishant Nikhil, Dyut Kumar Sil, Sumit Negi, Aruna Rajan

Cartography Active Learning (2021.findings-emnlp)

Challenge:	e-commerce and social media sites require content moderation to ensure ethical standards . a tiered moderation workflow with automated components complements human experts .
Approach:	They propose techniques for training text classification models under resource constraints . they use weak supervision, curriculum learning and multi-lingual training to fine-tune BERT .
Outcome:	The proposed techniques detect adversarial ads with a substantial gain over baseline . the authors show that the proposed methods can be applied to multiple languages .

Copied to clipboard

Mike Zhang, Barbara Plank

Bag of Tricks for In-Distribution Calibration of Pretrained Transformers (2023.findings-eacl)

Challenge:	Existing methods to label data are limited in their notion of informativeness, due to post-training model uncertainty and batch diversity.
Approach:	They propose a new Active Learning algorithm that exploits the behavior of the model on individual instances during training as a proxy to find the most informative instances for labeling.
Outcome:	The proposed method is competitive to other common AL methods, showing that training dynamics derived from small seed data can be successfully used for AL.

Copied to clipboard

Jaeyoung Kim, Dongbin Na, Sungchul Choi, Sungbin Lim

CASN:Class-Aware Score Network for Textual Adversarial Detection (2023.acl-long)

Challenge:	Recent studies show that pre-trained language models (PLMs) often predict over-confidently.
Approach:	They propose to use ensemble learning and data augmentation to improve confidence calibration for PLMs by combining calibration techniques with a trade-off between accuracy and classification.
Outcome:	The proposed calibration method improves classification accuracy and confidence in pre-trained language models by combining several calibration techniques.

Copied to clipboard

Rong Bao, Rui Zheng, Liang Ding, Qi Zhang, Dacheng Tao

Hierarchical Attention Prototypical Networks for Few-Shot Text Classification (D19-1)

Challenge:	Existing approaches to counteract adversarial attacks can be divided into two directions, adversarials defense and adversarially detection.
Approach:	They propose a score-based generative method to implicitly model the data distribution using a log-density distribution and supervised contrastive learning to guide the estimation using label information.
Outcome:	The proposed method improves on three text classification tasks on four advanced attack algorithms.

Copied to clipboard

Shengli Sun, Qingfeng Sun, Kevin Zhou, Tengchao Lv

Easy Adaptation to Mitigate Gender Bias in Multilingual Text Classification (2022.naacl-main)

Challenge:	Existing methods for text classification are based on large-scale labeled data, but few data are available.
Approach:	They propose a hierarchical attention prototypical networks for few-shot text classification . they use attention mechanism to highlight or weaken the importance of features, words, and instances .
Outcome:	The proposed model can capture more important features, words, and instances . it can also increase support set augmentability and accelerate convergence speed in training stage .

Copied to clipboard

Xiaolei Huang

Multilinguality

Challenge:	Existing approaches to mitigate demographic biases evaluate on monolingual data, however, multilingual data has not been examined.
Approach:	They propose a standard domain adaptation model to reduce gender bias in multilingual contexts.
Outcome:	The proposed model reduces gender bias and improves on two text classification tasks with three fair-aware baselines.

Meta-learning via Language Model In-context Tuning (2022.acl-long)

Copied to clipboard

Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, He He

BARLE: Background-Aware Representation Learning for Background Shift Out-of-Distribution Detection (2022.findings-emnlp)

Challenge:	Recent advances in large language models have reduced "task learning and prediction" to a simple sequence prediction problem.
Approach:	They propose a meta-learning method that recasts task adaptation and prediction as a sequence prediction problem.
Outcome:	The proposed method outperforms MAML on two classification tasks and improves on binaryClfs.

Copied to clipboard

Hanyu Duan, Yi Yang, Ahmed Abbasi, Kar Yan Tam

AutoIntent: AutoML for Text Classification (2025.emnlp-demos)

Challenge:	Existing methods for OOD detection focus on identifying semantic-shift OOD samples . background-shift detection is more practical but challenging .
Approach:	They propose a background-aware representation learning approach for background-shift OOD detection in NLP.
Outcome:	The proposed method improves background-shift OOD detection while maintaining ID classification accuracy.

Copied to clipboard

Ilya Alekseev, Roman Solomatin, Darina Rustamova, Denis Kuznetsov

Sensitivity as a Complexity Measure for Sequence Classification Tasks (2021.tacl-1)

Challenge:	Existing solutions for text classification tasks lack comprehensive support for hyperparameter optimization.
Approach:	They propose to automate text classification tasks using a modular, sklearn-like interface.
Outcome:	The proposed framework shows superior performance on intent classification datasets and enables users to balance effectiveness and resource consumption.

Copied to clipboard

Michael Hahn, Dan Jurafsky, Richard Futrell

Self-supervised Post-processing Method to Enrich Pretrained Word Vectors (2023.findings-emnlp)

Challenge:	Existing complexity metrics provide limited practical insight into complexity differences between tasks.
Approach:	They propose a theoretical framework for understanding and predicting the complexity of sequence classification tasks using a new extension of the theory of Boolean function sensitivity.
Outcome:	The proposed framework predicts the complexity of sequence classification tasks using a new method . it shows that low-sensitivity functions are easier to learn for LSTMs than lexical classifiers .

Copied to clipboard

Hwiyeol Jo

DropMix: A Textual Data Augmentation Combining Dropout with Mixup (2022.emnlp-main)

Challenge:	Existing methods that use external resources to make word vectors specialize depend on the lexicon.
Approach:	They propose a self-supervised extension of extrofitting by its own word vector distribution.
Outcome:	The proposed method improves word similarity embeddings on similarity tasks without external resources.

Copied to clipboard

Fanshuang Kong, Richong Zhang, Xiaohui Guo, Samuel Mensah, Yongyi Mao

Small Language Models in the Real World: Insights from Industrial Text Classification (2025.acl-industry)

Challenge:	Existing methods to overcome overfitting in text learning do not consider dimensionality . dimensionalization is important for deep neural networks to overcome the problem .
Approach:	They propose a saliency map-based approach to overcome overfitting in text learning . they propose augmentation regularization methods such as Dropout and Mixup to improve regularization .
Outcome:	Empirical results show that the proposed approach overcomes overfitting in text learning . dropout and mixup methods are effective in enhancing regularization .

Copied to clipboard

Lujun Li, Lama Sleem, Niccolo’ Gentile, Geoffrey Nichil, Radu State

Adv-OLM: Generating Textual Adversaries via OLM (2021.eacl-main)

Challenge:	With the emergence of ChatGPT, transformer-only models have significantly advanced text classification and related tasks.
Approach:	They propose to use prompt engineering and supervised fine-tuning methods for transformer-based text classification in industrial applications.
Outcome:	The proposed models perform well in a variety of industrial scenarios, including email classification, legal document categorization, and the classification of extremely long academic texts.

Copied to clipboard

Vijit Malik, Ashwani Bhat, Ashutosh Modi

Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations (N18-2)

Challenge:	Recent studies have pointed out the vulnerability of deep learning models to adversarial attacks.
Approach:	They propose a black-box attack method that adapts the idea of Occlusion and Language Models to the current state of the art attack methods.
Outcome:	The proposed method outperforms existing methods on several text classification tasks.

Copied to clipboard

Sosuke Kobayashi

What Did I Do Wrong? Quantifying LLMs’ Sensitivity and Consistency to Prompt Engineering (2025.naacl-long)

Challenge:	Neural network-based models for NLP have been growing with state-of-the-art results in various tasks.
Approach:	They propose a data augmentation method for labeled sentences called contextual augmentation.
Outcome:	The proposed method improves classifiers based on convolutional or recurrent neural networks.

Copied to clipboard

Federico Errica, Davide Sanvito, Giuseppe Siracusano, Roberto Bifulco

Zero-Shot Text Classification with Self-Training (2022.emnlp-main)

Challenge:	Large Language Models (LLMs) have significantly improved productivity in a number of routine tasks.
Approach:	They propose two metrics for classification tasks, namely sensitivity and consistency, which are complementary to task performance.
Outcome:	The proposed metrics are complementary to task performance and can be used to guide prompt engineering and obtain LLMs that balance robustness and performance.

Copied to clipboard

Ariel Gera, Alon Halfon, Eyal Shnarch, Yotam Perlitz, Liat Ein-Dor, Noam Slonim

Hierarchical Information Matters: Text Classification via Tree Based Graph Neural Network (2022.coling-1)

Challenge:	Recent advances in large pretrained language models have increased attention to zero-shot text classification.
Approach:	They propose a plug-and-play method to bridge this gap by requiring only class names along with an unlabeled dataset.
Outcome:	The proposed model can be trained on a natural language inference dataset and performs on dozens of unseen tasks without the need for domain expertise or trial and error.

Copied to clipboard

Chong Zhang, He Zhu, Xingyu Peng, Junran Wu, Ke Xu

Automated essay scoring with string kernels and word embeddings (P18-2)

Challenge:	Text classification is a primary task in natural language processing (NLP).
Approach:	They propose a graph neural network (HINT) that makes full use of hierarchical information contained in the text for the task of text classification.
Outcome:	The proposed method outperforms the state-of-the-art methods on popular benchmarks while having a simple structure and few parameters.

Copied to clipboard

Mădălina Cozma, Andrei Butnaru, Radu Tudor Ionescu

Representation Learning

Challenge:	Existing approaches to automatic essay scoring use low-level character n-gram features.
Approach:	They propose to combine string kernels and word embeddings for automatic essay scoring.
Outcome:	The proposed method outperforms state-of-the-art deep learning methods in Arabic dialect identification and native language identification tasks.

SelfMix: Robust Learning against Textual Label Noise with Self-Mixup Training (2022.coling-1)

Copied to clipboard

Dan Qiao, Chenchen Dai, Yuyang Ding, Juntao Li, Qiang Chen, Wenliang Chen, Min Zhang

Explaining Classes through Stable Word Attributions (2022.findings-acl)

Challenge:	Existing methods to handle label noise in text classification tasks are limited to visual data.
Approach:	They propose a method to handle label noise in text classification tasks using a Gaussian Mixture Model.
Outcome:	The proposed method outperforms baselines on three types of text classification tasks on visual and textual data.

Copied to clipboard

Samuel Rönnqvist, Aki-Juhani Kyröläinen, Amanda Myntti, Filip Ginter, Veronika Laippala

Certified Robustness to Word Substitution Attack with Differential Privacy (2021.naacl-main)

Challenge:	Input saliency methods have become popular for explaining predictions of deep learning models, but there has been little work investigating methods for aggregating prediction-level explanations to the class level.
Approach:	They propose a method to aggregate prediction-level explanations to the class level using XLM-R and Integrated Gradients input attribution methods.
Outcome:	The proposed method extracts keyword lists of classes from text classification tasks and evaluates them on web register data.

Copied to clipboard

Wenjie Wang, Pengfei Tang, Jian Lou, Li Xiong

More Identifiable yet Equally Performant Transformers for Text Classification (2021.acl-long)

Challenge:	Recent studies have shown that adversarial examples can be easily fooled by DNNs, making the robustness and security of NLP models significantly important.
Approach:	They propose a differential privacy-based algorithm to achieve certified robustness against word substitution at- tacks in text classification via differential privacy.
Outcome:	The proposed model achieves higher accuracy and more than 30X efficiency improvement over existing defense algorithms.

Copied to clipboard

Rishabh Bhardwaj, Navonil Majumder, Soujanya Poria, Eduard Hovy

Cross-lingual Text Classification Transfer: The Case of Ukrainian (2025.coling-main)

Challenge:	Current studies prove that attention weights are not unique and therefore unfit for interpretation.
Approach:	They propose a transformer encoder layer that decouples the relationship between key and value vector and provides identifiable weights up to the desired length of the input.
Outcome:	The proposed model is more identifiable than previously thought but still prone to be non-unique attentions that make them unfit for interpretation.

Copied to clipboard

Daryna Dementieva, Valeriia Khylenko, Georg Groh

HyperText: Endowing FastText with Hyperbolic Geometry (2020.findings-emnlp)

Challenge:	despite the large amount of labeled datasets, there is an imbalance in data availability across languages.
Approach:	They explore cross-lingual knowledge transfer methods avoiding manual data curation . they use large multilingual encoders and translation systems, LLMs, and language adapters .
Outcome:	The proposed approaches are tested on three text classification tasks in Ukrainian . the authors show that the proposed approaches avoid manual data curation .

Copied to clipboard

Yudong Zhu, Di Zhou, Jinghui Xiao, Xin Jiang, Xiao Chen, Qun Liu

Robust Text Classifier on Test-Time Budgets (D19-1)

Challenge:	Empirically, we show that HyperText outperforms FastText on a range of text classification tasks with much reduced parameters.
Approach:	They propose a model that uses hyperbolic geometry to model tree-like hierarchies in natural language sentences by embedding words or ngrams in hyperbolical space.
Outcome:	Empirically, the proposed model outperforms FastText on a range of text classification tasks with much reduced parameters.

Copied to clipboard

Md Rizwan Parvez, Tolga Bolukbasi, Kai-Wei Chang, Venkatesh Saligrama

Learning Label Modular Prompts for Text Classification in the Wild (2022.emnlp-main)

Challenge:	Recent advances in deep neural networks (DNNs) achieve high accuracy on many text classification tasks.
Approach:	They propose a generic framework for learning a robust text classification model . they use a data aggregation method to train the classifier on a large corpus of text .
Outcome:	The proposed framework achieves consistent speedup with little degradation in accuracy on four benchmark text classification tasks.

Copied to clipboard

Hailin Chen, Amrita Saha, Shafiq Joty, Steven C.H. Hoi

Zero-Shot Text Classification via Self-Supervised Tuning (2023.findings-acl)

Challenge:	Recent advances in parameter efficient tuning of pretrained language models have limited performance.
Approach:	They propose a label-modular prompt tuning framework for text classification tasks that emulates the transient nature of real-world.
Outcome:	The proposed framework outperforms baselines in two formidable settings and shows strong generalisation ability.

Copied to clipboard

Chaoqun Liu, Wenxuan Zhang, Guizhen Chen, Xiaobao Wu, Anh Tuan Luu, Chip Hong Chang, Lidong Bing

Multinomial Adversarial Networks for Multi-Domain Text Classification (N18-1)

Challenge:	Existing solutions to zero-shot text classification use pre-trained language models or large-scale annotated data.
Approach:	They propose a self-supervised learning paradigm to solve zero-shot text classification tasks by tuning the language models with unlabeled data.
Outcome:	The proposed model outperforms the state-of-the-art models on 7 out of 10 tasks and is less sensitive to prompt design.

Copied to clipboard

Xilun Chen, Claire Cardie

Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation (2023.acl-long)

Challenge:	Existing methods for text classification are domain-dependent, but there is no annotated data for some domains.
Approach:	They propose a multinomial adversarial network to tackle multi-domain text classification . they show that MANs significantly outperform prior art on the MDTC task .
Outcome:	The proposed model outperforms the prior art on the multi-domain text classification task.

Copied to clipboard

Yuxin Ren, Zihan Zhong, Xingjian Shi, Yi Zhu, Chun Yuan, Mu Li

Incorporating Latent Meanings of Morphological Compositions to Enhance Word Embeddings (P18-1)

Challenge:	Recent success of natural language processing (NLP) is driven by the adoption of large-scale pretrained language models.
Approach:	They propose a method to determine the impact of distillation influence on student generalization ability by prioritizing samples likely to enhance the student's generalization abilities.
Outcome:	The proposed method outperforms 10 common knowledge distillation baselines on 6 text classification tasks in the GLUE benchmark.

Copied to clipboard

Yang Xu, Jiawei Liu, Wei Yang, Liusheng Huang

Constructing contrastive samples via summarization for text classification with limited annotations (2021.findings-emnlp)

Challenge:	Existing word embedding methods learn semantic information at word level while neglecting meaningful inner structures of words like morphemes.
Approach:	They propose to use latent meanings of morphological compositions of words to train word embeddings.
Outcome:	The proposed models outperform baseline models on word similarity, syntactic analogy and text classification tasks.

Copied to clipboard

Yangkai Du, Tengfei Ma, Lingfei Wu, Fangli Xu, Xuhong Zhang, Bo Long, Shouling Ji

Strings from the Library of Babel: Random Sampling as a Strong Baseline for Prompt Optimisation (2024.naacl-long)

Challenge:	Various contrastive learning methods have been developed and lead to state-of-the-art performance in many computer vision tasks.
Approach:	They propose a method to construct efficient contrastive samples using text summarization to gain better representations of text classification tasks with limited annotations.
Outcome:	The proposed framework gains better representations on text classification tasks with limited annotations and is compared with existing methods on real-world text classification datasets.

Copied to clipboard

Yao Lu, Jiayi Wang, Raphael Tang, Sebastian Riedel, Pontus Stenetorp

CRL-Prompt: Contrastive and Reinforcement Learning for Soft Prompt Tuning for Text Classification (2026.acl-srw)

Challenge:	Recent prompt optimisation approaches use the generative nature of language models to produce prompts, even rivaling the performance of human-curated prompts.
Approach:	They propose to randomly sample tokens from the model vocabulary as "separators" they show that random separators are competitive baselines, having less than a 1% difference compared to previous self-optimisation methods.
Outcome:	The proposed method outperforms human-curated prompts in nine text classification tasks and eight language models and has a 40% chance of performing better than human-generated separators.

Copied to clipboard

Danila Lapokin, Andrey Savchenko

Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training (2021.emnlp-main)

Challenge:	Manual prompt engineering is time-consuming, non-scalable, and brittle, while current auto-prompting techniques are far from maturity.
Approach:	They propose a two-stage method for prompt learning of frozen language models, CRL-Prompt, based on soft prompt initialization followed by contrastive and reinforcement-based refinement.
Outcome:	The proposed method achieves consistent improvements over baseline prompt tuning strategies, with gains of up to 2.2% while training fewer than 0.25% of model parameters.

Copied to clipboard

Kuan-Hao Huang, Wasi Ahmad, Nanyun Peng, Kai-Wei Chang

Balancing via Generation for Multi-Class Text Classification Improvement (2020.findings-emnlp)

Challenge:	Pre-trained multilingual language encoders do not precisely align words and phrases across languages.
Approach:	They propose a learning strategy for training robust models by drawing connections between adversarial examples and failure cases of zero-shot cross-lingual transfer.
Outcome:	The proposed model can achieve good performance even if representations of different languages are not aligned well.

Copied to clipboard

Naama Tepper, Esther Goldbraich, Naama Zwerdling, George Kour, Ateret Anaby Tavor, Boaz Carmeli

Improving the results of string kernels in sentiment analysis and Arabic dialect identification by adapting them to your test set (D18-1)

Challenge:	balancing is a known technique for improving classification performance . balancy is based on a balancing policy and a text generation mechanism .
Approach:	They propose a balancing-via-generation framework that augments a dataset for more balanced distribution by using a text generation mechanism.
Outcome:	The proposed framework can augment a dataset for more balanced distribution while under-sampling others.

Copied to clipboard

Radu Tudor Ionescu, Andrei M. Butnaru

Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation (2025.acl-long)

Challenge:	Recent studies have demonstrated remarkable performance in text classification tasks such as Arabic dialect identification.
Approach:	They propose two approaches to improve string kernels' accuracy in Arabic and English . first approach interprets pairwise string kernel similarities between training and test sets as features . second approach adapts to training set and adds test samples for another round of training .
Outcome:	The proposed methods improve English polarity classification and Arabic dialect identification.

Copied to clipboard

Mingxuan Xia, Haobo Wang, Yixuan Li, Zewei Yu, Jindong Wang, Junbo Zhao, Runze Wu

IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models (2025.naacl-long)

Challenge:	Existing methods for data annotation use an aggressive approach prompting LLMs to determine a single gold label for each unlabeled sample.
Approach:	They propose a teacher-student framework that distills candidate annotations with a Small Language Model (SLM) they propose to use LLMs to generate and distill candidate annotation with slms to ensure unique labels are provided for downstream tasks.
Outcome:	The proposed method outperforms existing methods due to uncertainty in LLMs and is noisetolerant.

Copied to clipboard

David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba Oluwadara Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En-Shiun Annie Lee, Chiamaka Ijeoma Chukwuneke, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Foutse Yuehgoh, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Salomey Osei, Shamsuddeen Hassan Muhammad, Sokhar Samb, Tadesse Kebede Guge, Tombekai Vangoni Sherman, Pontus Stenetorp

Multilinguality

Challenge:	Large language models (LLMs) are limited to a few high-resource languages . many low-resourced languages are evaluated only on basic text classification tasks .
Approach:	They propose to use IrokoBench to evaluate 17 low-resource African languages . they use human-translated benchmark datasets to evaluate zero-shot, few-shot and translate-test settings .
Outcome:	The proposed model performs well in English and French, but the highest performing model perform poorly in proprietary models.

Vector of Locally Aggregated Embeddings for Text Representation (N19-1)

Copied to clipboard

Hadi Amiri, Mitra Mohtarami

Representation Learning

Challenge:	Existing models for text classification use word embeddings, weighted averaging, and deepening networks.
Approach:	They propose a vector-based locally averaging model that encodes each input text by effectively identifying and integrating its semantically-relevant parts.
Outcome:	The proposed model outperforms RNNs and CNNs in text classification while taking only a fraction of their training time.

Generative Calibration for In-context Learning (2023.findings-emnlp)

Copied to clipboard

Zhongtao Jiang, Yuanzhe Zhang, Cao Liu, Jun Zhao, Kang Liu

Hierarchical Verbalizer for Few-Shot Hierarchical Text Classification (2023.acl-long)

Challenge:	In-context learning is one of the most exciting features of large language models . performance is sensitive to various configurations of the prompt, such as the choice or order of the training examples.
Approach:	They propose to calibrate the in-context predictive distribution by adjusting the label marginal . they find that the proposed method outperforms the ICL and state-of-the-art calibration methods .
Outcome:	The proposed method outperforms state-of-the-art methods by 27% absolute in macro-F1.

Copied to clipboard

Ke Ji, Yixin Lian, Jingsheng Gao, Baoyuan Wang

Adaptive Learning of Local Semantic and Global Structure Representations for Text Classification (C18-1)

Challenge:	Existing work on the hierarchical text classification problem is limited due to the complexity of label hierarchy and intensive labeling cost.
Approach:	They propose a path-based few-shot setting and a strict path-basic evaluation metric to further explore few- shot HTC tasks.
Outcome:	The proposed framework outperforms those who inject hierarchy through graph encoders on three popular HTC datasets under the few-shot setting.

Copied to clipboard

Jianyu Zhao, Zhiqiang Zhan, Qichuan Yang, Yang Zhang, Changjian Hu, Zhensheng Li, Liuxin Zhang, Zhiqiang He

Your fairness may vary: Pretrained language model fairness in toxic text classification (2022.findings-acl)

Challenge:	Existing representation models for text classification learn little structure information or rely on pre-defined structures.
Approach:	They propose a sandwich neural network to learn local semantic and global structure representations without relying on parsers.
Outcome:	The proposed approach achieves competitive performance on several text classification tasks.

Copied to clipboard

Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Moninder Singh, Mikhail Yurochkin

Scheduled DropHead: A Regularization Method for Transformer Models (2020.findings-emnlp)

Challenge:	Pre-trained, bidirectional language models have revolutionized natural language processing research . authors show that focusing on accuracy measures alone can lead to models with wide variation in fairness characteristics .
Approach:	They propose to use two post-processing methods to improve model fairness without retraining . they use pretrained language models of varying sizes on two toxic text classification tasks .
Outcome:	The proposed methods improve model fairness without retraining . the results show that the fairness variation is more than just accuracy .

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou, Ke Xu

Devil’s Advocate: Novel Boosting Ensemble Method from Psychological Findings for Text Classification (2021.findings-emnlp)

Challenge:	DropHead is a structured dropout method for regularizing multi-head attention . DropHed drops entire attention heads during training to prevent overfitting .
Approach:	They propose a structured dropout method specifically designed for regularizing multi-head attention mechanism . DropHead drops entire attention heads during training to prevent overfitting .
Outcome:	The proposed method can improve transformer models by 0.9 BLEU score on translation task and around 1.0 accuracy for various text classification tasks.

Copied to clipboard

Hwiyeol Jo, Jaeseo Lim, Byoung-Tak Zhang

Is BERT a Cross-Disciplinary Knowledge Learner? A Surprising Finding of Pre-trained Models’ Transferability (2021.findings-emnlp)

Challenge:	Existing ensemble methods that combine submodels to create a composite model can improve model performance by diminishing model bias and variance.
Approach:	They propose a method which uses a deliberately dissenting model to force other submodels within the ensemble to better collaborate.
Outcome:	The proposed method shows comparable or improved performance on 5 text classification tasks when compared to conventional methods.

Copied to clipboard

Wei-Tsung Kao, Hung-yi Lee

ZeroAE: Pre-trained Language Model based Autoencoder for Transductive Zero-shot Text Classification (2023.findings-acl)

Challenge:	Using pre-trained language models, we can apply them to specialized domains such as scientific articles or clinical data.
Approach:	They propose to pre-train BERT models on large text corpora and use them to generalize to token sequence classification applications.
Outcome:	The models pre-trained on text classification tasks perform better than the models using task-specific knowledge and share non-trivial similarities.

Copied to clipboard

Kaihao Guo, Hang Yu, Cong Liao, Jianguo Li, Haipeng Zhang

To Pretrain or Not to Pretrain: Examining the Benefits of Pretrainng on Resource Rich Tasks (2020.acl-main)

Challenge:	Existing methods for text classification use only encoders or decoders that do not allow for the use of labels in unseen domains.
Approach:	They propose an autoencoder that encodes text into two disentangled spaces and decodes it to generate text with labels in the unseen domains.
Outcome:	The proposed model outperforms the existing methods in label-partially-unseen and label-fully-un-seeen scenarios and even outperfects the SOTA methods.

Copied to clipboard

Sinong Wang, Madian Khabsa, Hao Ma

Performance-Guided LLM Knowledge Distillation for Efficient Text Classification at Scale (2024.emnlp-main)

Challenge:	Existing studies on pretraining NLP models with variants of Masked Language Model (MLM) objectives have shown that the number of training samples used in the downstream task is limited.
Approach:	They propose to use MLM objectives to pretrain NLP models with variants of Masked Language Model (MLM) objectives to improve accuracy on downstream tasks.
Outcome:	The proposed model can reach a diminishing return point as the supervised data size increases significantly.

Copied to clipboard

Flavio Palo, Prateek Singhi, Bilal Fadlallah

Fighting Spurious Correlations in Text Classification via a Causal Learning Perspective (2025.naacl-long)

Challenge:	Large Language Models (LLMs) face high computational demands at inference time due to high computational costs.
Approach:	They propose a cost-effective and high-throughput solution for large language models . PGKD distills the knowledge of LLMs into smaller, task-specific models based on teacher-student knowledge distillation .
Outcome:	PGKD outperforms BERT-based models and other knowledge distillation methods on multi-class classification datasets.

Copied to clipboard

Yuqing Zhou, Ziwei Zhu

Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning (2024.findings-naacl)

Challenge:	In text classification tasks, models often rely on spurious correlations for predictions, incorrectly associating irrelevant features with the target labels.
Approach:	They propose a Causally Calibrated Robust Classifier which integrates a causal feature selection method based on counterfactual reasoning and an unbiased inverse propensity weighting (IPW) loss function.
Outcome:	The proposed method achieves state-of-the-art performance among methods without group labels and can compete with the models that utilize group labels.

Copied to clipboard

Shuai Zhao, Leilei Gan, Anh Tuan Luu, Jie Fu, Lingjuan Lyu, Meihuizi Jia, Jinming Wen

On Evaluation Protocols for Data Augmentation in a Limited Data Scenario (2025.coling-main)

Challenge:	Existing methods for parameter-efficient fine-tuning (PEFT) are not effective for weight-poisoning backdoor attacks.
Approach:	They propose a parameter-efficient fine-tuning (PEFT) method that updates only a limited set of model parameters and provides a robust defense against weight-poisoning backdoor attacks.
Outcome:	The proposed method identifies poisoned samples through confidence and is robust against weight-poisoning backdoor attacks.

Copied to clipboard

Frédéric Piedboeuf, Philippe Langlais

Information Aggregation via Dynamic Routing for Sequence Encoding (C18-1)

Challenge:	Textual data augmentation (DA) is a prolific field of study where novel techniques to create artificial data are regularly proposed.
Approach:	They propose to use textual data augmentation (DA) to generate new sentences for text classification in a limited data setting.
Outcome:	The proposed methods perform better on small data settings and on large datasets, but they are not as effective on large data sets.

Copied to clipboard

Jingjing Gong, Xipeng Qiu, Shaojing Wang, Xuanjing Huang

Multimodality

Challenge:	Currently, little attention is paid to how to aggregate text sequences into fixed-size vectors.
Approach:	They propose an aggregation mechanism to obtain a fixed-size encoding with a dynamic routing policy.
Outcome:	The proposed method outperforms other aggregating methods on five text classification tasks.

Lifelong Explainer for Lifelong Learners (2021.emnlp-main)

Copied to clipboard

Xuelin Situ, Sameen Maruf, Ingrid Zukerman, Cecile Paris, Gholamreza Haffari

Lifelong Language Knowledge Distillation (2020.emnlp-main)

Challenge:	Existing explanation methods are inefficient when explaining a static black-box model.
Approach:	They propose a Lifelong Explanation approach that continuously trains a student explainer under the supervision of a teacher on different tasks undertaken in LL.
Outcome:	The proposed approach can be extended to include a teacher and maintain the same level of faithfulness to the black-box model as the student explainer while being up to 102 times faster at test time.

Copied to clipboard

Yung-Sung Chuang, Shang-Yu Su, Yun-Nung Chen

AEDA: An Easier Data Augmentation Technique for Text Classification (2021.findings-emnlp)

Challenge:	Existing methods to perform lifelong language learning (LLL) on stream of different tasks are challenging . Existing models face catastrophic forgetting problem, which can be mitigated by lifelong learning .
Approach:	They propose a method that can be easily applied to existing LLL architectures to mitigate degradation.
Outcome:	The proposed method improves state-of-the-art models and reduces degradation compared to multi-task models.

Copied to clipboard

Akbar Karimi, Leonardo Rossi, Andrea Prati

GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network (2024.findings-acl)

Challenge:	AEDA is an easier data augmentation technique than EDA.
Approach:	They propose an augmentation technique that includes only random insertion of punctuation marks into the original text.
Outcome:	The proposed method is easier to implement for data augmentation than EDA method.

Copied to clipboard

Shuzhou Yuan, Ercong Nie, Michael Färber, Helmut Schmid, Hinrich Schuetze

Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning (2021.emnlp-main)

Challenge:	Large Language Models (LLMs) exhibit strong In-Context Learning (ICL) capabilities when prompts with demonstrations are used.
Approach:	They propose a prompt-based parameter-efficient fine-tuning approach that leverages insights into ICL’s information flow dynamics and hardwires the desired information flow into the GNN.
Outcome:	The proposed approach surpasses prompt-based fine-tuning methods in few-shot settings by updating just 0.2% to 0.5% of parameters.

Copied to clipboard

Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, Xipeng Qiu

ProFormer: Towards On-Device LSH Projection Based Transformers (2021.eacl-main)

Challenge:	Pre-trained models can be maliciously poisoned with certain triggers, causing a security threat.
Approach:	They propose a stronger weight poisoning attack method that introduces a layerwise weight poison strategy to plant deeper backdoors.
Outcome:	The proposed method can be widely applied and provide hints for future models robustness studies.

Copied to clipboard

Chinnadhurai Sankar, Sujith Ravi, Zornitsa Kozareva

PlugAT: A Plug and Play Module to Defend against Textual Adversarial Attack (2022.coling-1)

Challenge:	ProFormer is a projection based transformer architecture that is faster and lighter making it suitable to deploy to memory constraint devices such as mobile phones, watches and IoT.
Approach:	They propose a projection based transformer architecture that generates word representations on-the-fly without embedding lookup tables and a local projection attention layer that transforms the input sequence of N LSH word projections into a sequence of K representations.
Outcome:	The proposed architecture reduces memory footprint from 92.16 MB to 1.7 KB and requires 16x less computation overhead making it suitable to deploy to memory constraint devices and preserve user privacy.

Copied to clipboard

Rui Zheng, Rong Bao, Qin Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Rui Xie, Wei Wu

FENAS: Flexible and Expressive Neural Architecture Search (2020.findings-emnlp)

Challenge:	Recent studies have shown that adversarial examples can be easily fooled by adversarially perturbed examples.
Approach:	They propose a pluggable defense module PlugAT to provide robust predictions by adding a few trainable parameters to the model inputs while keeping the original model frozen.
Outcome:	The proposed model improves robustness over several strong baselines whilst training only 9.1% parameters.

Copied to clipboard

Ramakanth Pasunuru, Mohit Bansal

Knowledge Distillation ≈ Label Smoothing: Fact or Fallacy? (2023.emnlp-main)

Challenge:	Recent approaches to architecture search have shown good improvements in terms of performance with reasonable training speed.
Approach:	They propose an algorithm with more activation functions, input edges, and atomic operations to search for architectures that are optimal for given task.
Outcome:	The proposed algorithm reproduces well-known LSTM and GRU architectures and initializes with them for finding architectures more efficiently.

Copied to clipboard

Md Sultan

Beyond the Tip of the Iceberg: Assessing Coherence of Text Classifiers (2021.findings-emnlp)

Challenge:	Knowledge distillation (KD) is a method for knowledge transfer from one model to another . recent studies suggest it is based on label smoothing, but it is not .
Approach:	They propose to compare the predictive confidences of models trained with knowledge distillation . they propose to use a method that is similar to label smoothing to train models .
Outcome:	Experiments on four text classification tasks show that knowledge distillation and label smoothing drive model confidence in opposite directions.

Copied to clipboard

Shane Storks, Joyce Chai

The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance (2024.findings-acl)

Challenge:	Large-scale, pre-trained language models achieve human-level and superhuman accuracy on existing language understanding tasks, but statistical bias in benchmark data and probing studies has recently called into question their true capabilities.
Approach:	They propose to evaluate systems through a measure of prediction coherence by using two existing language understanding benchmarks with different properties to demonstrate its versatility.
Outcome:	The proposed evaluation framework is quick, effective, and versatile to provide insight into the coherence of machines’ predictions.

Copied to clipboard

Abel Salinas, Fred Morstatter

RANCC: Rationalizing Neural Networks via Concept Clustering (2020.coling-main)

Challenge:	Large Language Models (LLMs) are used to label data across many domains and for myriad tasks.
Approach:	They ask large language models to label data using a series of decisions by practitioners . they find that even the smallest perturbations can change the LLM's answer .
Outcome:	The proposed model can be used to quickly get a response for arbitrary tasks.

Copied to clipboard

Housam Khalifa Bashier, Mi-Young Kim, Randy Goebel

Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification (2021.findings-acl)

Challenge:	Existing models that construct explanations concurrently with classification predictions are opaque.
Approach:	They propose a self-explainable model for Natural Language Processing (NLP) text classification tasks . they extract a rationale from the text and use it to predict a concept of interest .
Outcome:	The proposed model can be compressed without complicated compression techniques.

Copied to clipboard

Yada Pruksachatkun, Satyapriya Krishna, Jwala Dhamala, Rahul Gupta, Kai-Wei Chang

Towards Intention Understanding in Suicidal Risk Assessment with Natural Language Processing (2022.findings-emnlp)

Challenge:	Existing methods to reduce disparities in model outcomes have focused on data augmentation, debiasing model embeddings, or adding fairness-based optimization objectives during training.
Approach:	They propose to use certified word substitution robustness methods to improve equality of odds and equality of opportunity on multiple text classification tasks.
Outcome:	The proposed methods improve equality of odds and equality of opportunity on multiple text classification tasks.

Copied to clipboard

Shaoxiong Ji

On the Impact of Temporal Concept Drift on Model Explanations (2022.findings-emnlp)

Challenge:	Suicide is a global problem, with one suicide case for every 100 deaths worldwide . social networking sites are an essential forum for communication and information sharing .
Approach:	This paper compares natural language processing to suicidal ideation detection and risk assessment . it urges better intention understanding for reliable suicide risk assessment with computational methods .
Outcome:	This paper compares the performance of natural language processing to suicidal ideation detection and risk assessment tasks.

Copied to clipboard

Zhixue Zhao, George Chrysostomou, Kalina Bontcheva, Nikolaos Aletras

Parameter-Efficient Language Model Tuning with Active Learning in Low-Resource Settings (2023.emnlp-main)

Challenge:	Explanation faithfulness of model predictions is typically evaluated on held-out data from the same temporal distribution as the training data.
Approach:	They examine the impact of temporal variation on model explanations extracted by eight feature attribution methods and three select-then-predict models across six text classification tasks.
Outcome:	The proposed method shows the most robust faithfulness scores across datasets and in asynchronous settings.

Copied to clipboard

Josip Jukić, Jan Snajder

Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax (2021.findings-acl)

Challenge:	Pre-trained language models (PLMs) have ignited a surge in demand for effective fine-tuning techniques . data labeling is notoriously time-consuming and expensive, hindering the development of sizable labeled datasets .
Approach:	They propose to use active learning to reduce labeling costs by minimizing label complexity . they find PEFT adapter modules have significant potential in low-resource settings .
Outcome:	The proposed model outperforms FFT in low-resource settings and shows that it yields more stable representations of early and middle layers than FFT.

Copied to clipboard

Ehsan Kamalloo, Mehdi Rezagholizadeh, Peyman Passban, Ali Ghodsi

Data Quality Enhancement on the Basis of Diversity with Large Language Models for Text Classification: Uncovered, Difficult, and Noisy (2025.coling-main)

Challenge:	Existing kNN-based augmentation techniques blindly incorporate all samples, but MiniMax-kNN uses a subset of augmented samples to maximize KL-divergence between teacher and student models.
Approach:	They propose a semi-supervised approach to augmented data augmentation using kNN.
Outcome:	The proposed method outperforms existing kNN-based augmentation techniques on several classification tasks and requires fewer augmented examples and less computation to achieve superior performance.

Copied to clipboard

Min Zeng, Caiquan Liu, Shiqi Zhang, Li Xie, Chen Sang, Xiaoxin Chen

Efficiently Tuned Parameters Are Task Embeddings (2022.emnlp-main)

Challenge:	Existing methods for text classification based on large language models are difficult to apply directly to solve.
Approach:	They propose a data quality enhancement method to improve LLMs' performance in classification tasks by using a greedy algorithm to select data and then performing fine-tuning.
Outcome:	The proposed method improves the performance of large language models in text classification tasks and significantly improves training efficiency, saving nearly half of the training time.

Copied to clipboard

Wangchunshu Zhou, Canwen Xu, Julian McAuley

Seeing Both the Forest and the Trees: Multi-head Attention for Joint Classification on Different Compositional Levels (2020.coling-main)

Challenge:	Existing methods for intermediate-task transfer are computationally infeasible to experiment with all intermediate combinations.
Approach:	They propose to use task-specific parameters updated in parameter-efficient tuning methods to predict inter-task transferability.
Outcome:	The proposed approach outperforms existing methods while being conceptually simple and computationally efficient.

Copied to clipboard

Miruna Pislar, Marek Rei

D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Bias (2023.findings-acl)

Challenge:	Neural networks can capture expressive language features, but insights into the link between words and sentences are difficult to acquire automatically.
Approach:	They propose a deep neural network architecture that explicitly wires lower and higher linguistic components and evaluate its ability to perform the same task at different hierarchical levels.
Outcome:	The proposed model outperforms equivalent models that are not incentivized towards compositional representations.

Copied to clipboard

Sabit Hassan, Malihe Alikhani

Leveraging Training Dynamics and Self-Training for Text Classification (2022.findings-emnlp)

Challenge:	Infusing clustering with active learning with AL can overcome the bias issue of both AL and traditional annotation methods while exploiting AL’s annotation efficiency.
Approach:	They propose an algorithm that dynamically adjusts clustering and annotation efforts in response to an estimated classifier error-rate.
Outcome:	The proposed algorithm outperforms baseline AL approaches with pretrained transformers and traditional Support Vector Machines on eight datasets for emotion, hatespeech, dialog act, and book type detection tasks.

Copied to clipboard

Tiberiu Sosea, Cornelia Caragea

Correcting Language Model Bias for Text Classification in True Zero-Shot Learning (2024.lrec-main)

Challenge:	Semi-supervised learning (SSL) is a promising technique for improving deep learning models when training data is scarce.
Approach:	They propose a semi-supervised learning approach that leverages training dynamics of unlabeled data.
Outcome:	The proposed method achieves an average increase in F1 score of 3.5% over baselines in low resource settings.

Copied to clipboard

Feng Zhao, Wan Xianlin, Cheng Yan, Chu Kiong Loo

Meta-training with Demonstration Retrieval for Efficient Few-shot Learning (2023.findings-acl)

Challenge:	Experimental results show that pre-trained language models outperform standard prompt learning in zero-shot settings.
Approach:	They propose a pipeline for annotating and filtering examples from unlabeled examples . they propose 'model bias validation' method that utilizes unlabed examples as validation set .
Outcome:	The proposed approach outperforms standard prompt learning on six text classification tasks.

Copied to clipboard

Aaron Mueller, Kanika Narang, Lambert Mathias, Qifan Wang, Hamed Firooz

Why and when should you pool? Analyzing Pooling in Recurrent Architectures (2020.findings-emnlp)

Challenge:	Large language models have impressive fewshot performance on many NLP tasks and domains.
Approach:	They propose a meta-training approach that uses demonstration retrieval to train parameter-efficient models that generalize well on a larger variety of tasks.
Outcome:	The proposed approach outperforms many parameter-efficient methods on QA, NLI, and text classification tasks.

Copied to clipboard

Pratyush Maini, Keshav Kolluru, Danish Pruthi, null Mausam

Topics to Avoid: Demoting Latent Confounds in Text Classification (D19-1)

Challenge:	Various pooling techniques have been shown to improve performance of RNNs on text classification tasks.
Approach:	They propose a pooling-based variant that captures interactions among predictive tokens in a sentence.
Outcome:	The proposed pooling architecture outperforms non-pooling models on sequence classification tasks.

Copied to clipboard

Sachin Kumar, Shuly Wintner, Noah A. Smith, Yulia Tsvetkov

Controlling Bias Exposure for Fair Interpretable Predictions (2022.findings-emnlp)

Challenge:	Despite impressive performance on many text classification tasks, deep neural networks tend to learn frequent superficial patterns that are specific to the training data and do not always generalize well.
Approach:	They propose a method that represents latent topical confounds and a model which “unlearns” confounding features by predicting both the label of the input text and the confound.
Outcome:	The proposed model generalizes better and learns features indicative of the writing style rather than the content.

Copied to clipboard

Zexue He, Yu Wang, Julian McAuley, Bodhisattwa Prasad Majumder

Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning (2021.naacl-main)

Challenge:	Existing approaches to reduce bias in NLP tasks focus on protecting or isolating information related to a sensitive attribute, but they lack control over how much bias is required to be removed.
Approach:	They propose a favorable debiasing method that uses sensitive information ‘fairly’, rather than blindly eliminating it.
Outcome:	The proposed method achieves a trade-off between debiasing and task performance along with producing debiased rationales as evidence.

Copied to clipboard

Jason Wei, Chengyu Huang, Soroush Vosoughi, Yu Cheng, Shiqi Xu

Discriminative Language Model as Semantic Consistency Scorer for Prompt-based Few-Shot Text Classification (2024.lrec-main)

Challenge:	a few-shot text classification task requires a large number of output classes, with few training examples per class.
Approach:	They propose a data augmentation technique suitable for training with limited data for few-shot, highly-multiclass text classification scenarios.
Outcome:	The proposed technique improves performance on four classification tasks by 3.0% on average.

Copied to clipboard

Zhipeng Xie, Yahe Li

Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks (2020.coling-main)

Challenge:	Existing methods for prompt-based finetuning are expensive and difficult to use.
Approach:	They propose a prompt-based method that uses a pretrained language model to discriminate between original and replacement tokens.
Outcome:	The proposed method outperforms state-of-the-art prompt-based few-shot methods on 10 widely-used text classification tasks.

Copied to clipboard

Trapit Bansal, Rishikesh Jha, Andrew McCallum

Continual Few-Shot Learning for Text Classification (2021.emnlp-main)

Challenge:	Pre-trained transformer models have shown great success in improving performance on downstream tasks, but fine-tuning on a new task still requires large amounts of labeled data.
Approach:	They propose a method which allows optimization-based meta-learning across tasks . they use transformers to train transformer models and find better generalizations .
Outcome:	The proposed method outperforms self-supervised training and pre-trained models on 17 NLP tasks.

Copied to clipboard

Ramakanth Pasunuru, Veselin Stoyanov, Mohit Bansal

UOR: Universal Backdoor Attacks on Pre-trained Language Models (2024.findings-acl)

Challenge:	a large number of end-to-end systems are needed for many tasks in natural language processing.
Approach:	They propose a continual few-shot learning task where a system is asked to correct mistakes with a few training examples.
Outcome:	The proposed task compares two NLI and one sentiment analysis datasets with baselines from diverse paradigms.

Copied to clipboard

Wei Du, Peixuan Li, Haodong Zhao, Tianjie Ju, Ge Ren, Gongshen Liu

Semantic matching for text classification with complex class descriptions (2023.emnlp-main)

Challenge:	Existing methods to attack pre-trained language models rely on manual selection of triggers and backdoor representations.
Approach:	They propose a backdoor attack method that turns manual selection into automatic optimization . they propose to use poisoned contrastive learning to learn more uniform backdoor representations .
Outcome:	The proposed method achieves better attack performance on text classification tasks compared to manual methods.

Copied to clipboard

Brian De Silva, Kuan-Wen Huang, Gwang Lee, Karen Hovsepian, Yan Xu, Mingwei Shen

Courtroom-LLM: A Legal-Inspired Multi-LLM Framework for Resolving Ambiguous Text Classifications (2025.coling-main)

Challenge:	Existing methods for text classification support zero-shot learning but not both . Existing approaches do not support zero or few-shot, and are insufficient for complex classes .
Approach:	They propose a method which rapidly adapts from seen classes to new/unseen ones . they use labels and complex class descriptions to perform zero- and few-shot learning .
Outcome:	The proposed method beats baselines on complex class descriptions by 22.48% . it also improves zero-shot learning by 4.29% .

Copied to clipboard

Sangkeun Jung, Jeesu Jung

HashFormers: Towards Vocabulary-independent Pre-trained Transformers (2022.emnlp-main)

Challenge:	Using a multi-LLM structure inspired by legal courtroom processes, we demonstrate that it can improve decision-making accuracy in ambiguous text classification scenarios.
Approach:	They propose a legal-inspired multi-LLM structure that simulates a courtroom setting within LLMs and assigns roles similar to those of prosecutors, defense attorneys, and judges.
Outcome:	The proposed model outperforms both single-LLM classifiers and simpler multi-LLMS setups in ambiguous text classification tasks.

Copied to clipboard

Huiyin Xue, Nikolaos Aletras

Less is More: Attention Supervision with Counterfactuals for Text Classification (2020.emnlp-main)

Challenge:	Existing pre-trained language models are vocabulary-dependent, mapping by default each token to its corresponding embedding.
Approach:	They propose a family of vocabulary-independent pre-trained transformers that support unlimited vocabulary . they propose to map each token to its corresponding embedding by default .
Outcome:	The proposed models are more memory efficient than existing models while achieving comparable performance on multiple text classification tasks.

Copied to clipboard

Seungtaek Choi, Haeju Park, Jinyoung Yeo, Seung-won Hwang

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity (2022.acl-long)

Challenge:	Specifically, we explore the advantage of counterfactual reasoning, over associative reasoning . Adding human supervision to attention has been shown to improve model predictions and explanations .
Approach:	They propose to use machine-augmented human attention supervision to enhance model quality.
Outcome:	The proposed method is more effective than existing methods requiring higher annotation cost . the proposed method can be trained to generate similar attention to human supervision .

Copied to clipboard

Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, Pontus Stenetorp

Uncertainty Estimation of Transformer Predictions for Misclassification Detection (2022.acl-long)

Challenge:	Large pretrained language models can generate text classification results that match fully supervised models.
Approach:	They propose to use a few sample training to determine which permutations are performant . they use generative language models to construct an artificial development set .
Outcome:	The proposed model outperforms fully-supervised models in eleven text classification tasks.

Copied to clipboard

Artem Vazhentsev, Gleb Kuzmin, Artem Shelmanov, Akim Tsvigun, Evgenii Tsymbalov, Kirill Fedyanin, Maxim Panov, Alexander Panchenko, Gleb Gusev, Mikhail Burtsev, Manvel Avetisian, Leonid Zhukov

Label Semantic Aware Pre-training for Few-shot Text Classification (2022.acl-long)

Challenge:	Uncertainty estimation (UE) of model predictions is crucial step for a variety of tasks such as active learning, misclassification detection, adversarial attack detection, etc.
Approach:	They propose to modify UE methods for Transformer models for misclassification detection in named entity recognition and text classification tasks to improve model expressiveness and computational performance.
Outcome:	The proposed methods outperform computationally intensive methods on misclassification detection tasks and are based on a large dataset of simulated datasets.

Copied to clipboard

Aaron Mueller, Jason Krone, Salvatore Romeo, Saab Mansour, Elman Mansimov, Yi Zhang, Dan Roth

Statistical Depth for Ranking and Characterizing Transformer-Based Text Embeddings (2023.emnlp-main)

Challenge:	Existing models for text classification use label semantics but few studies have attempted to give models access to informative representations of labels.
Approach:	They propose to use label semantics to train generative models by performing secondary pre-training on labeled sentences from a variety of domains.
Outcome:	The proposed approach improves generalization and data efficiency of text classification systems while maintaining comparable performance to state-of-the-art models.

Copied to clipboard

Parker Seegmiller, Sarah Preum

Exploring the Usability of Persuasion Techniques for Downstream Misinformation-related Classification Tasks (2024.lrec-main)

Challenge:	Generalized transformer-based text embedding models have produced state of the art performance results on a variety of tasks such as natural language inference (NLI)
Approach:	They propose a statistical depth to measure distributions of transformer-based text embeddings and an associated rank sum test to characterize distributions in synthetic and human-generated corpora.
Outcome:	The proposed method improves performance over baseline methods on six text classification tasks.

Copied to clipboard

Nikolaos Nikolaidis, Jakub Piskorski, Nicolas Stefanovitch

Hybrid Uncertainty Quantification for Selective Text Classification in Ambiguous Tasks (2023.acl-long)

Challenge:	systematically explore the predictive power of features derived from Persuasion Techniques detected in texts for different tasks of interest for media analysis.
Approach:	They propose a set of meaningful features aimed at capturing persuasiveness of a text . they also assess the discriminatory power of these features in different text classification tasks .
Outcome:	The proposed features can be applied to detecting mis/disinformation, fake news, propaganda, partisan news and conspiracy theories.

Copied to clipboard

Artem Vazhentsev, Gleb Kuzmin, Akim Tsvigun, Alexander Panchenko, Maxim Panov, Mikhail Burtsev, Artem Shelmanov

LightFormer: Light-weight Transformer Using SVD-based Weight Transfer and Parameter Sharing (2023.findings-acl)

Challenge:	Existing methods for text classification tasks are inherently ambiguous and can cause errors.
Approach:	They propose a method that combines epistemic and aleatoric uncertainty to estimate toxicity detection errors.
Outcome:	The proposed method outperforms existing methods for toxicity detection and other ambiguous text classification tasks.

Copied to clipboard

Xiuqing Lv, Peng Zhang, Sunzhu Li, Guobing Gan, Yueheng Sun

GAProtoNet: A Multi-head Graph Attention-based Prototypical Network for Interpretable Text Classification (2025.coling-main)

Challenge:	Deploying Transformer networks on resource-constrained edge devices is challenging.
Approach:	They propose a low-rank factorization initialized by SVD-based weight transfer and parameter sharing to compress and accelerate Transformer networks.
Outcome:	The proposed method achieves similar performance to the baseline Transformer with 3.8 times and 1.8 times fewer parameters and achieves 2.3 times speedup and 1.5 times speed up respectively.

Copied to clipboard

Ximing Wen, Wenjuan Tan, Rosina Weber

What Do Language Models Learn in Context? The Structured Task Hypothesis. (2024.acl-long)

Challenge:	Existing models with black-box nature obscure decision-making process and lack interpretability.
Approach:	They propose a multi-head graph attention-based prototypical network that uses a vector and prototypes to learn an interpretable prototypical representation.
Outcome:	The proposed model achieves superior results without sacrificing the accuracy of the original black-box LMs.

Copied to clipboard

Jiaoda Li, Yifan Hou, Mrinmaya Sachan, Ryan Cotterell

EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks (D19-1)

Challenge:	Pre-trained large language models have exhibited an impressive ability to learn in context across various domains, e.g., code generation, education, medicine and even medicine.
Approach:	They taxonomize existing candidate theories into three competing hypotheses that explain LLMs’ ability to learn in context.
Outcome:	The proposed model can learn a task from in-context examples presented in a demonstration and generalize it to the prompt.

Copied to clipboard

Jason Wei, Kai Zou

Explainable Text Classification with LLMs: Enhancing Performance through Dialectical Prompting and Explanation-Guided Training (2025.findings-emnlp)

Challenge:	Existing data augmentation techniques for text classification are difficult to implement and cost a high amount of money.
Approach:	They propose to use four simple but powerful operations to boost performance on text classification tasks to improve synonym replacement, random insertion, random swap, and random deletion.
Outcome:	The proposed techniques improve performance on five classification tasks and are particularly useful for smaller datasets.

Copied to clipboard

Huaming Du, Lei Yuan, Cancan Feng, Guisong Liu, Gang Kou, Carl Yang

Detecting Adversarial Samples through Sharpness of Loss Landscape (2023.findings-acl)

Challenge:	Existing explanation methods that generate keywords may be less effective due to missing critical contextual information.
Approach:	They propose a new method to generate explanations for possible labels using LLMs and a dialectical prompt.
Outcome:	The proposed method significantly improves accuracy and explanation quality over state-of-the-art methods on multiple datasets from diverse domains.

Copied to clipboard

Rui Zheng, Shihan Dou, Yuhao Zhou, Qin Liu, Tao Gui, Qi Zhang, Zhongyu Wei, Xuanjing Huang, Menghan Zhang

Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation (2020.emnlp-main)

Challenge:	Existing studies have shown that adversarial samples are more vulnerable than normal ones to textual adversarials.
Approach:	They propose a simple and effective sharpness-based detector that can distinguish adversarial samples by maximizing the loss increment within the region where the inference sample is located.
Outcome:	The proposed method outperforms previous detection methods by large margins on three text classification tasks.

Copied to clipboard

Ruibo Liu, Guangxuan Xu, Chenyan Jia, Weicheng Ma, Lili Wang, Soroush Vosoughi

Beyond Text: Leveraging Multi-Task Learning and Cognitive Appraisal Theory for Post-Purchase Intention Analysis (2024.findings-acl)

Challenge:	Existing methods for data augmentation produce low readability or semantic consistency.
Approach:	They propose a framework which augments data through reinforcement learning guided conditional generation.
Outcome:	The proposed framework improves F1 performance on three different classification tasks by 8.7% on average when given only 10% of the whole data for training.

Copied to clipboard

Gerard Yeo, Shaz Furniturewala, Kokil Jaidka

Robust Text Classification: Analyzing Prototype-Based Networks (2024.findings-emnlp)

Challenge:	Recent studies have shown that user-level features can carry more task-related information than the text itself.
Approach:	They evaluate multi-task learning frameworks grounded in Cognitive Appraisal Theory to predict user behavior as a function of users’ self-expression and psychological attributes.
Outcome:	The proposed models improve on the language and traits of users, while lacking rich annotations of other attributes.

Copied to clipboard

Zhivar Sourati, Darshan Deshpande, Filip Ilievski, Kiril Gashteovski, Sascha Saralajew

Target-to-Source Augmentation for Aspect Sentiment Triplet Extraction (2023.emnlp-main)

Challenge:	Language models exhibit a drop in performance on noisy data, which can cause classifiers to incorrectly change their predictions.
Approach:	They propose to use Prototype-Based Networks to classify examples based on their similarity to prototypical examples of a class (prototypes) they show that PBNs offer more robustness under both targeted and static adversarial attacks.
Outcome:	The proposed model is robust to noise and targets both targeted and static attacks.

Copied to clipboard

Yice Zhang, Yifan Yang, Meng Li, Bin Liang, Shiwei Chen, Ruifeng Xu

Sentiment Analysis

Challenge:	Aspect Sentiment Triplet Extraction (ASTE) is an important task in sentiment analysis, but data scarcity limits performance of existing methods.
Approach:	They propose a target-to-source augmentation approach to alleviate the issue of data scarcity in Aspect Sentiment Triplet Extraction (ASTE) they use fluency and alignment discriminators to provide feedback and use this feedback to optimize the generator.
Outcome:	The proposed approach significantly improves the performance of existing methods.

All You Need is Attention: Lightweight Attention-based Data Augmentation for Text Classification (2024.findings-emnlp)

Copied to clipboard

Junehyung Kim, Sungjae Hwang

SeqVAT: Virtual Adversarial Training for Semi-Supervised Sequence Labeling (2020.acl-main)

Challenge:	Existing methods to augment text classification tasks require extensive dataset training.
Approach:	They propose a method that uses attention mechanisms to exchange semantically similar words between sentences to generate a greater diversity of synthetic sentences compared to simpler operations like random insertions.
Outcome:	The proposed method consistently outperforms baseline methods across diverse text classification conditions.

Copied to clipboard

Luoxin Chen, Weitong Ruan, Xinyue Liu, Jianhua Lu

Robustifying Sentiment Classification by Maximally Exploiting Few Counterfactuals (2022.emnlp-main)

Challenge:	Empirical studies show that virtual adversarial training (VAT) significantly improves the sequence labeling performance over baselines under supervised and semi-supervised settings.
Approach:	They propose a method which naturally applies VAT to sequence labeling models with conditional random field (CRF) Empirical studies show that SeqVAT significantly improves the sequence labelling performance over baselines under supervised settings, and outperforms state-of-the-art approaches under semi-supervised settings.
Outcome:	Empirical results show that the proposed method outperforms state-of-the-art approaches under semi-supervised settings.

Copied to clipboard

Maarten De Raedt, Fréderic Godin, Chris Develder, Thomas Demeester

On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection (2023.findings-acl)

Challenge:	a recent study found that finetuned language models rely on spurious patterns in training data . this limitation limits their performance on out-of-distribution (OOD) test data.
Approach:	They propose a method that only requires annotation of a small fraction of training data . they add 1% manual counterfactuals to training data and generate extra counterfacts in vector space .
Outcome:	The proposed approach improves sentiment classification using IMDb data and other sets for OOD tests.

Copied to clipboard

SongYang Gao, Shihan Dou, Qi Zhang, Xuanjing Huang, Jin Ma, Ying Shan

Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning (2023.findings-emnlp)

Challenge:	Existing adversarial detection methods require access to training data, which brings noteworthy concerns regarding privacy leakage and generalizability.
Approach:	They propose a data-agnostic adversarial detection framework which induces different responses between normal and adversarials to UAPs.
Outcome:	The proposed framework achieves competitive detection performance on various text classification tasks, and maintains equivalent time consumption to normal inference.

Copied to clipboard

Zhe Yang, Damai Dai, Peiyi Wang, Zhifang Sui

InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective (2023.findings-emnlp)

Challenge:	Large Language Models (LLMs) have recently gained the In-Context Learning ability . however, the quality of demonstration examples is usually uneven .
Approach:	They propose to determine optimal weights for demonstration examples and apply them during ICL.
Outcome:	The proposed approach outperforms conventional ICL on 8 classification tasks.

Copied to clipboard

Yifan Song, Peiyi Wang, Weimin Xiong, Dawei Zhu, Tianyu Liu, Zhifang Sui, Sujian Li

Vulnerability of LLMs to Vertically Aligned Text Manipulations (2025.acl-long)

Challenge:	Recent studies have identified the severe performance decrease on analogous classes as a key factor for catastrophic forgetting.
Approach:	They propose a replay-based continual text classification method that uses fast-slow and current-past contrastive learning to perform mutual information maximization and better recover previously learned representations.
Outcome:	The proposed method achieves state-of-the-art on three text classification tasks.

Copied to clipboard

Zhecheng Li, Yiwei Wang, Bryan Hooi, Yujun Cai, Zhen Xiong, Nanyun Peng, Kai-Wei Chang

Do BERT-Like Bidirectional Models Still Perform Better on Text Classification in the Era of LLMs? (2025.findings-emnlp)

Challenge:	Recent research shows that vertical text input significantly degrades the accuracy of large language models (LLMs) in text classification tasks.
Approach:	They investigate the impact of vertical text input on the performance of LLMs . they find that chain of thought reasoning does not help LLM recognize vertical input .
Outcome:	The proposed model can significantly mislead models, posing a risk of bypassing detection in real-world scenarios involving harmful or sensitive information.

Copied to clipboard

Junyan Zhang, Yiming Huang, Shuliang Liu, Yubo Gao, Xuming Hu

On an Intermediate Task for Classifying URL Citations on Scholarly Papers (2024.lrec-main)

Challenge:	Rapid adoption of LLMs has overshadowed the potential advantages of traditional BERT-like models in text classification.
Approach:	They compare BERT-like models fine-tuning, LLM internal state utilization, and LLM zero-shot inference across six datasets.
Outcome:	The proposed method outperforms LLMs on six challenging datasets.

Copied to clipboard

Kazuhiro Wada, Masaya Tsunokake, Shigeki Matsubara

GAttention: Gated Attention for the Detection of Abusive Language (2025.findings-emnlp)

Challenge:	Citations using URLs can be used as information source for research resource search engines.
Approach:	They propose a method to classify URL citations using a simple fine-tuning strategy.
Outcome:	The proposed method outperforms methods using a simple fine-tuning strategy with higher macro F-scores for different model sizes and architectures.

Copied to clipboard

Horacio Jarquín Vásquez, Hugo Jair Escalante, Manuel Montes, Mario Ezra Aragon

GottBERT: a pure German Language Model (2024.emnlp-main)

Challenge:	Abusive language online creates toxic environments and exacerbates social tensions, underscoring the need for robust NLP models to interpret nuanced linguistic cues.
Approach:	They propose a Gated Attention mechanism that combines the strengths of Contextual attention and Self-attention mechanisms to address the limitations of existing attention models within the text classification task.
Outcome:	The novel gated attention mechanism addresses the limitations of existing attention models within the text classification task.

Copied to clipboard

Raphael Scheible, Johann Frei, Fabian Thomczyk, Henry He, Patric Tippmann, Jochen Knaus, Victor Jaravine, Frank Kramer, Martin Boeker

Adversarial Text Generation using Large Language Models for Dementia Detection (2024.emnlp-main)

Challenge:	Pre-trained language models have advanced natural language processing (NLP) despite the introduction of BERT, single-language models are still relevant.
Approach:	They present a German singlelanguage RoBERT model pre-trained exclusively on the German portion of the OSCAR dataset.
Outcome:	The GottBERT model outperforms the existing models on Named Entity Recognition and text classification tasks.

Copied to clipboard

Youxiang Zhu, Nana Lin, Kiran Balivada, Daniel Haehn, Xiaohui Liang

Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets (2025.acl-long)

Challenge:	Large language models excel in text classification tasks, but they do not perform well with picture description.
Approach:	They propose an interpretable classification approach by Adversarial Text Generation (ATG) that could relate dementia detection with other tasks.
Outcome:	The proposed approach achieves 85% accuracy, >10% improvement over the previous methods.

Copied to clipboard

Dongyue Li, Ziniu Zhang, Lu Wang, Hongyang R. Zhang

Toward Efficient Sparse Autoencoder-Guided Steering for Improved In-Context Learning in Large Language Models (2025.emnlp-main)

Challenge:	Existing methods for fine-tuning language models are efficient when adapting to a single dataset.
Approach:	They propose to use an ensemble method for fine-tuning a language model to multiple datasets instead of a single adapter per task.
Outcome:	The proposed method improves performance on multiple datasets while preserving low-rank adaptation properties.

Copied to clipboard

Ikhyun Cho, Julia Hockenmaier

LLM-Guided Semantic Bootstrapping for Interpretable Text Classification with Tsetlin Machines (2026.findings-acl)

Challenge:	Sparse autoencoders (SAEs) have emerged as a powerful analytical tool in mechanistic interpretability for large language models (LLMs).
Approach:	They propose a novel approach that leverages SAEs to enhance the general in-context learning performance of large language models (LLMs).
Outcome:	The proposed method yields a 3.5% improvement across diverse text classification tasks and exhibits greater robustness to hyperparameter variations compared to standard steering approaches.

Copied to clipboard

Jiechao Gao, Rohan Kumar Yadav, Yuangang Li, Yuandong Pan, Jie Wang, Ying Liu, Michael Lepech

VI-OOD: A Unified Framework of Representation Learning for Textual Out-of-distribution Detection (2024.lrec-main)

Challenge:	Pretrained language models (PLMs) provide strong semantic representations but are costly and opaque.
Approach:	They propose a framework that transfers pretrained language models into symbolic form and integrates them into symbolic models.
Outcome:	The proposed framework improves interpretability and accuracy across multiple text classification tasks while remaining fully symbolic and efficient.

Copied to clipboard

Li-Ming Zhan, Bo Liu, Xiao-Ming Wu